Deep Reinforcement Learning for Wind and Energy Storage Coordination in Wholesale Energy and Ancillary Service Markets

Global power systems are increasingly reliant on wind energy as a mitigation strategy for climate change. However, the variability of wind energy causes system reliability to erode, resulting in the wind being curtailed and, ultimately, leading to substantial economic losses for wind farm owners. Wind curtailment can be reduced using battery energy storage systems (BESS) that serve as onsite backup sources. Yet, this auxiliary role may significantly hamper the BESS's capacity to generate revenues from the electricity market, particularly in conducting energy arbitrage in the Spot market and providing frequency control ancillary services (FCAS) in the FCAS markets. Ideal BESS scheduling should effectively balance the BESS's role in absorbing onsite wind curtailment and trading in the electricity market, but it is difficult in practice because of the underlying coordination complexity and the stochastic nature of energy prices and wind generation. In this study, we investigate the bidding strategy of a wind-battery system co-located and participating simultaneously in both the Spot and Regulation FCAS markets. We propose a deep reinforcement learning (DRL)-based approach that decouples the market participation of the wind-battery system into two related Markov decision processes for each facility, enabling the BESS to absorb onsite wind curtailment while simultaneously bidding in the wholesale Spot and FCAS markets to maximize overall operational revenues. Using realistic wind farm data, we validated the coordinated bidding strategy for the wind-battery system and find that our strategy generates significantly higher revenue and responds better to wind curtailment compared to an optimization-based benchmark. Our results show that joint-market bidding can significantly improve the financial performance of wind-battery systems compared to individual market participation

Optimal Energy Storage Scheduling for Wind Curtailment Reduction and Energy Arbitrage: A Deep Reinforcement Learning Approach

Wind energy has been rapidly gaining popularity as a means for combating climate change. However, the variable nature of wind generation can undermine system reliability and lead to wind curtailment, causing substantial economic losses to wind power producers. Battery energy storage systems (BESS) that serve as onsite backup sources are among the solutions to mitigate wind curtailment. However, such an auxiliary role of the BESS might severely weaken its economic viability. This paper addresses the issue by proposing joint wind curtailment reduction and energy arbitrage for the BESS. We decouple the market participation of the co-located wind-battery system and develop a joint-bidding framework for the wind farm and BESS. It is challenging to optimize the joint-bidding because of the stochasticity of energy prices and wind generation. Therefore, we leverage deep reinforcement learning to maximize the overall revenue from the spot market while unlocking the BESS's potential in concurrently reducing wind curtailment and conducting energy arbitrage. We validate the proposed strategy using realistic wind farm data and demonstrate that our joint-bidding strategy responds better to wind curtailment and generates higher revenues than the optimization-based benchmark. Our simulations also reveal that the extra wind generation used to be curtailed can be an effective power source to charge the BESS, resulting in additional financial returns.Comment: 2023 IEEE Power & Energy Society General Meeting (PESGM). arXiv admin note: text overlap with arXiv:2212.1336

Model-Free Approach to Fair Solar PV Curtailment Using Reinforcement Learning

The rapid adoption of residential solar photovoltaics (PV) has resulted in regular overvoltage events, due to correlated reverse power flows. Currently, PV inverters prevent damage to electronics by curtailing energy production in response to overvoltage. However, this disproportionately affects households at the far end of the feeder, leading to an unfair allocation of the potential value of energy produced. Globally optimizing for fair curtailment requires accurate feeder parameters, which are often unknown. This paper investigates reinforcement learning, which gradually optimizes a fair PV curtailment strategy by interacting with the system. We evaluate six fairness metrics on how well they can be learned compared to an optimal solution oracle. We show that all definitions permit efficient learning, suggesting that reinforcement learning is a promising approach to achieving both safe and fair PV coordination

NCL++: Nested Collaborative Learning for Long-Tailed Visual Recognition

Long-tailed visual recognition has received increasing attention in recent years. Due to the extremely imbalanced data distribution in long-tailed learning, the learning process shows great uncertainties. For example, the predictions of different experts on the same image vary remarkably despite the same training settings. To alleviate the uncertainty, we propose a Nested Collaborative Learning (NCL++) which tackles the long-tailed learning problem by a collaborative learning. To be specific, the collaborative learning consists of two folds, namely inter-expert collaborative learning (InterCL) and intra-expert collaborative learning (IntraCL). In-terCL learns multiple experts collaboratively and concurrently, aiming to transfer the knowledge among different experts. IntraCL is similar to InterCL, but it aims to conduct the collaborative learning on multiple augmented copies of the same image within the single expert. To achieve the collaborative learning in long-tailed learning, the balanced online distillation is proposed to force the consistent predictions among different experts and augmented copies, which reduces the learning uncertainties. Moreover, in order to improve the meticulous distinguishing ability on the confusing categories, we further propose a Hard Category Mining (HCM), which selects the negative categories with high predicted scores as the hard categories. Then, the collaborative learning is formulated in a nested way, in which the learning is conducted on not just all categories from a full perspective but some hard categories from a partial perspective. Extensive experiments manifest the superiority of our method with outperforming the state-of-the-art whether with using a single model or an ensemble. The code will be publicly released.Comment: arXiv admin note: text overlap with arXiv:2203.1535

A novel image integration technology mapping system significantly reduces radiation exposure during ablation for a wide spectrum of tachyarrhythmias in children

ObjectiveRadiofrequency catheter ablation (RFCA) has evolved into an effective and safe technique for the treatment of tachyarrhythmia in children. Concerns about children and involved medical staff being exposed to radiation during the procedure should not be ignored. “Fluoroscopy integrated 3D mapping”, a new 3D non-fluoroscopic navigation system software (CARTO Univu Module) could reduce fluoroscopy during the procedure. However, there are few studies about the use of this new technology on children. In the present study, we analyzed the impact of the CARTO Univu on procedural safety and fluoroscopy in a wide spectrum of tachyarrhythmias as compared with CARTO3 alone.MethodsThe data of children with tachyarrhythmias who underwent RFCA from June 2018 to December 2021 were collected. The CARTO Univu was used for mapping and ablation in 200 cases (C3U group) [boys/girls (105/95), mean age (6.8 ± 3.7 years), mean body weight (29.4 ± 7.9 kg)], and the CARTO3 was used in 200 cases as the control group (C3 group) [male/female (103/97), mean age (7.2 ± 3.9 years), mean body weight (32.3 ± 19.0 kg)]. The arrhythmias were atrioventricular reentrant tachycardia (AVRT, n = 78), atrioventricular node reentrant tachycardia (AVNRT, n = 35), typical atrial flutter (AFL, n = 12), atrial tachycardia (AT, n = 20) and ventricular arrhythmias [VAs, premature ventricular complexes or ventricular tachycardia, n = 55].Results① There was no significant difference in the acute success rate, recurrence rate, and complication rate between the C3 and C3U groups [(94.5% vs. 95.0%); (6.3% vs. 5.3%); and (2.0% vs. 1.5%); P > 0.05]. ② The CARTO Univu reduced radiation exposure: fluoroscopy time: AVRT C3: 8.5 ± 7.2 min vs. C3U: 4.5 ± 2.9 min, P < 0.05; AVNRT C3: 10.7 ± 3.2 min vs. C3U: 4.3 ± 2.6 min, P < 0.05; AT C3: 15.7 ± 8.2 min vs. C3U: 4.5 ± 1.7 min, P < 0.05; AFL C3: 8.7 ± 3.2 min vs. C3U: 3.7 ± 2.7 min, P < 0.05; VAs C3: 7.7 ± 4.2 min vs. C3U: 3.9 ± 2.3 min, P < 0.05. Corresponding to the fluoroscopy time, the fluoroscopy dose was also reduced significantly. ③ In the C3U group, the fluoroscopy during VAs ablation was lower than that of other arrhythmias (P < 0.05).ConclusionThe usage of the “novel image integration technology” CARTO Univu might be safe and effective in RFCA for a wide spectrum of tachyarrhythmias in children, which could significantly reduce fluoroscopy and has a more prominent advantage for VAs ablation